Investigating the validity of subjective workload rating (NASA TLX) and subjective situation awareness rating (SART) for cognitively complex human–machine work

Subjective workload and situation awareness measures, such as the NASA task load index (TLX) situational rating technique (SART), are frequently used in human–system evaluation. However, interpretation of these ratings is debated. In this study, empirical evidence for measures' theoretical assumptions was investigated by comparing operators' collected immediately after performing a scenario acquisition through video review scenario, knowledge actual system states. Eighteen licensed control room operators participated simulator running 12 relatively challenging scenarios. It found that TLX items involving introspection remained stable acquired factual knowledge, while perception external events, performance, depended on knowledge. The result shows operators’ could discriminate between mental effort, frustration, awareness. No clear SART measure found. Instead, subjective developed study distinct from related to operator showing type warrants future investigation its validity. findings help developing measurement procedures interpreting measures. Finally, reveals informing about can provide useful performance. Future research should include how inform participants adequately efficiently assessments.